The invention relates generally to the field of recording and generating images, and more particularly to the generation and displaying of panoramic images stereoscopically. The invention specifically provides a system and method for generating a stereoscopic panoramic image pair, comprising respective left and right panoramic images, in which disparity is controlled to provide selected effects.
When a person views a scene with his or her eyes, the disparity, or angular difference in viewing directions by his or her eyes for each point in the scene, is interpreted by the brain as depth, or distance, to an object in the scene through a process referred to as xe2x80x9cstereo fusion.xe2x80x9d The disparity is a function of both the distance of the object from the viewer and a baseline corresponding to, for example, the distance between the viewer""s eyes, which results in an angle that the brain interprets as distance. Depth can be perceived if the disparity is in a particular angular range. If an object is too far away from a person, the disparity would be too small for depth to be perceived. The maximum stereoscopic range, however, can be extended by use of aids such as binoculars, which, in addition to magnifying, can also serve to increase the baseline. On the other hand, if the object is too close to the viewer, the disparity would be too large for stereo fusion.
Depth can also be perceived from images that have been recorded of the scene. In order to observe depth, two images are required of the same portion of the scene, one of which will be viewed by the left eye and the other of which will be viewed by the right eye. A pair of such images, which will be referred to as a stereoscopic image pair, comprises two images, which have been recorded from two different viewpoints. Since the images comprising a stereoscopic image pair are recorded from two different viewpoints, they will provide a disparity, in a manner similar to the way in which the eyes provide disparity if they were viewing the scene directly. The amount of the disparity will depend on the distance of the points in the scene from the position at which the images were recorded and the distance between the viewpoints at which the images were recorded.
U.S. patent application Ser. No. 09/396,248, filed Sep. 16, 1999, in the names of Shmuel Peleg, et al., entitled xe2x80x9cSystem and Method for Generating and Displaying Panoramic Images and Moviesxe2x80x9d (hereinafter referred to as xe2x80x9cthe Peleg applicationxe2x80x9d) describes a system and method for generating images comprising a stereoscopic panoramic image pair, which can be viewed to provide a stereoscopic panoramic view of a scene. A stereoscopic panoramic image pair comprises two panoramic images, one of which can be viewed by the left eye and the other of which can be viewed by the right eye. In the system described in the Peleg application, each panoramic image in the stereoscopic panoramic image pair is generated from images recorded by a single camera rotated around an axis located behind the camera""s center of projection. The camera is rotated around the rotational axis in a series of steps, and an image is recorded at each step. Strips from successive images are mosaiced together to form mosaic panoramic images. Generally, two strips will be obtained from each image for use in generating the mosaic panoramic images, one to the right of the center of the image and the other to the left of the center of the image, with the strips being generally parallel to the axis of rotation of the camera. As described in the Peleg application, strips that are mosaiced together for the right panoramic image will be the strips that are to the left of the center of the image (as the image is viewed), and strips that are mosaiced together for the left panoramic image will be the strips that are to the right of the center of the image. The widths of the strips, and the angle between steps at which the camera records images, is selected to ensure that each pair of strips, that is, the pair of strips to the right of the center of the image, and the pair of strips to the left of the center of the image, can be mosaiced together to form respective right and left mosaic panoramic images of the stereoscopic panoramic image pair.
As noted above, two strips will be obtained from each image for use in generating the mosaic panoramic images, one to the right of the center of the image and the other to the left of the center of the image. Since the light rays enter the camera through a lens or pinhole, the light rays that are recorded to define the strip to the right of the center will be reflected off surfaces generally directed toward the right (as seen by the camera) of in the scene, and the light rays that are recorded to define the strip to the left of the center will be reflected off surfaces generally directed to the left (as seen by the camera) of in the scene. It will be appreciated that, in an image, the left and right strips will comprise images of different objects, but when the strips are mosaiced together to form the respective left and right panoramic images, the left and right panoramic images will contain images of the same objects, although from different, that is, left and right, viewpoints. When the portions of the left and right mosaic panoramic images that are images of the same portion of the scene are simultaneously displayed to the respective left and right eyes of a viewer, the viewer will be able to observe stereoscopic depth in the same manner as if he or she had viewed the scene with his or her eyes. It will be appreciated that the separation between the locations in the camera""s image plane at which the left and right strips are recorded corresponds to the baseline that can provide the disparity required to view depth stereoscopically. It will be appreciated that, for relatively distant objects, the disparity may be insufficient to allow for stereo fusion. Alternatively, depending on the variation of distance of objects in the scene from the camera, the disparity may vary widely.
The invention provides a new and improved system and method for generating a stereoscopic panoramic image pair, comprising respective left and right panoramic images, in which disparity is controlled to provide selected effects, which may include, for example, providing a minimum or maximum disparity for all objects in the scene to allow for stereo fusion therefor, adjusting disparity so to a selected range, or other effects that will be apparent to those skilled in the art.
In brief summary, the invention provides, in one aspect, a system for generating a stereoscopic panoramic image pair comprising left and right panoramic mosaic images for use in facilitating stereoscopic viewing of a panoramic scene, the system comprising a panoramic image generator and a strip separation value generation module. The panoramic image generator is configured to mosaic together respective left and right image strips from respective ones of a plurality of images to form the respective left and right panoramic images. The panoramic image generator is configured to use strip separation values for the respective images to determine the separation of the respective left and right image strips in generating the respective left and right panoramic images. The strip separation value generation module is configured to generate, for respective ones of said images, the strip separation values so as to provide at least one selected disparity for objects in the panoramic scene.
In another aspect, the invention provides a method generating a stereoscopic panoramic image pair comprising left and right panoramic mosaic images for use in facilitating stereoscopic viewing of a panoramic scene. In an initial panoramic image generation step, respective left and right image strips from respective ones of a plurality of images are mosaiced together to form the respective left and right panoramic images. During the initial panoramic image generation step a predetermined strip separation value being used to determine the separation of the respective left and right image strips in generating the respective left and right panoramic images. During an initial disparity value generation step, respective initial left and right panoramic images are processed to generate initial disparity values for projections of at least some points in the scene. During a separation value generation step, final strip separation values are generated from the initial disparity values. During a final initial panoramic image generation step, respective left and right image strips from respective ones of said plurality of images are mosaiced together to form the respective left and right panoramic images. During the final panoramic image generation step, the final strip separation values are used to determine the separation of the respective left and right image strips in generating the respective left and right panoramic images.
In yet another aspect, the invention provides a computer program product for use in connection with a programmable device to facilitate generation of a stereoscopic panoramic image pair comprising left and right panoramic mosaic images for use in facilitating stereoscopic viewing of a panoramic scene, the computer program product comprising a device-readable medium having encoded thereon a panoramic image generation module, an initial disparity value generation module, a separation value generation module, and a control module. The panoramic image generation module is configured to enable said programmable device to mosaic together respective left and right image strips from respective ones of a plurality of images to form the respective left and right panoramic images, the panoramic image generation module being enabled to use a selected strip separation value to determine the separation of the respective left and right image strips in generating the respective left and right panoramic images. The initial disparity value generation module is configured to enable said programmable device to process respective initial left and right panoramic images to generate initial disparity values for projections of at least some points in the scene. The separation value generation module is configured to enable said programmable device to generate final strip separation values from the initial disparity values. The control module is configured to
(i) enable programmable device to utilize said panoramic image generation module, in the process initially using a predetermined strip separation value in generating initial left and right panoramic images for use by said initial disparity value generation module, and
(ii) after the final strip generation values have been generated, enable the programmable device to utilize said panoramic image generation module, in the process using the final strip separation value in generating final left and right panoramic images.